{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "模型的部署参考： [learn-llm-deploy-easily](https://gitee.com/coderwillyan/learn-llm-deploy-easily) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这里主要介绍如何调用已部署的模型"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# rerank"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 什么是Reranker"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "\n",
    "Reranker是一种用于优化信息检索结果的深度学习模型，通过对初步检索（如关键词匹配或向量相似度检索）得到的候选文档进行二次排序，提升结果的相关性和准确性。其核心作用包括：  \n",
    "弥补初步检索的局限性：传统方法（如TF-IDF、BM25或Embedding相似度）难以捕捉深层次语义，Reranker通过交叉编码（Cross-Encoder）直接分析查询与文档的原始文本，实现细粒度匹配。  \n",
    "\n",
    "提升结果质量：通过多维度评估（语义一致性、上下文关联性等）对文档重新打分，确保高相关性内容优先展示。  \n",
    "\n",
    "### 工作原理与技术实现\n",
    "\n",
    "两阶段检索架构\n",
    "第一阶段（粗筛）：使用快速检索方法（如向量数据库ANN搜索）从海量数据中召回Top-K候选文档，侧重效率。  \n",
    "第二阶段（精排）：Reranker对候选文档进行精细排序，通常基于Transformer模型（如BERT、BGE等）计算查询-文档对的语义相似度分数。\n",
    "\n",
    "### 关键技术\n",
    "交叉编码（Cross-Encoder）：将查询和文档拼接为单一输入，通过自注意力机制直接计算相关性，避免双编码器（Bi-Encoder）的信息压缩问题。  \n",
    "损失函数优化：如局部对比估计（LCE）损失，增强模型区分相关与不相关文档的能力。  \n",
    "轻量化设计：通过梯度缓存、混合精度训练等技术降低GPU内存占用，支持大规模部署。  \n",
    "\n",
    "### 主要类型\n",
    "\n",
    "基于统计的Reranker：使用多路召回加权或倒数排名融合（RRF）算法，计算高效但语义理解较浅，适合延迟敏感场景。  \n",
    "\n",
    "基于深度学习的Reranker：  \n",
    "\n",
    "通用模型：如BGE系列（bge-reranker-base）、Qwen-Reranker，支持中英文和多语言任务。  \n",
    "\n",
    "领域专用模型：如针对电商、医疗等垂直领域微调的模型。  \n",
    "\n",
    "\n",
    "\n",
    "\n",
    "### 优势\n",
    "语义深度：优于Embedding模型的浅层相似度计算，支持歧义消解和隐含需求捕捉。\n",
    "\n",
    "模块化设计：独立于检索阶段，便于迭代优化（如领域适配微调）。\n",
    "\n",
    "### 挑战\n",
    "计算成本：在线推理需处理每个文档，延迟和费用显著高于向量检索（如重排Top-100文档成本增加5000倍）。\n",
    "\n",
    "性能权衡：需在检索质量、延迟和成本间平衡，高并发场景可能需轻量化方案（如FlashRank）。\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 使用reranker API"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.retrievers.document_compressors import CohereRerank\n",
    "import cohere\n",
    "from langchain_core.documents import Document\n",
    "\n",
    "cohere_client = cohere.Client(api_key=\"Tahx1eySFbKvu9sTyTXrRLf59la3ZUG9vy02stRZ\")\n",
    "\n",
    "compressor = CohereRerank(\n",
    "    client=cohere_client,\n",
    "    top_n=3,\n",
    "    model=\"rerank-multilingual-v3.0\"  # 支持多语言的新版本\n",
    ")\n",
    "\n",
    "# 测试样例数据\n",
    "documents = [\n",
    "    Document(page_content=\"巴黎是法国的首都，也是著名的艺术文化中心。\", metadata={\"source\": \"wiki\"}),\n",
    "    Document(page_content=\"北京是中国的政治和文化中心，拥有紫禁城等历史建筑。\", metadata={\"source\": \"gov\"}),\n",
    "    Document(page_content=\"Capital punishment refers to the death penalty in legal systems.\", metadata={\"source\": \"law\"}),\n",
    "    Document(page_content=\"东京是日本最大的城市，也是全球重要的经济中心。\", metadata={\"source\": \"news\"}),\n",
    "    Document(page_content=\"Washington D.C. is the capital of the United States.\", metadata={\"source\": \"edu\"}),\n",
    "    Document(page_content=\"首尔是韩国的首都，以现代化与传统文化的融合闻名。\", metadata={\"source\": \"travel\"}),\n",
    "    Document(page_content=\"Capitalization in finance refers to the total value of a company's shares.\", metadata={\"source\": \"finance\"})\n",
    "]\n",
    "\n",
    "query = \"各国首都城市的介绍有哪些？\"\n",
    "\n",
    "# 执行重排序\n",
    "compressed_docs = compressor.compress_documents(documents=documents, query=query)\n",
    "\n",
    "# 打印排序结果\n",
    "print(\"===== 重排序后的Top 5文档 =====\")\n",
    "for i, doc in enumerate(compressed_docs):\n",
    "    print(f\"Rank {i+1} (Score: {doc.metadata['relevance_score']:.4f}):\")\n",
    "    print(f\"内容：{doc.page_content}\")\n",
    "    print(f\"元数据：{doc.metadata}\\n{'-'*50}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_community.document_compressors import JinaRerank  # 使用Jina的rerank组件\n",
    "\n",
    "# Jina Rerank配置\n",
    "JINA_API_KEY = \"jina_63bb115e2d5f42d581f42643294792b5CE4nrEINMDcT4vJZJaSLcr5tkbIB\"  # 替换为你的Jina API密钥\n",
    "\n",
    "compressor = JinaRerank(\n",
    "    jina_api_key=JINA_API_KEY,\n",
    "    top_n=3,\n",
    "    model=\"jina-reranker-v2-base-multilingual\"  # Jina的多语言rerank模型[5](@ref)\n",
    ")\n",
    "\n",
    "# 测试样例数据\n",
    "documents = [\n",
    "    Document(page_content=\"巴黎是法国的首都，也是著名的艺术文化中心。\", metadata={\"source\": \"wiki\"}),\n",
    "    Document(page_content=\"北京是中国的政治和文化中心，拥有紫禁城等历史建筑。\", metadata={\"source\": \"gov\"}),\n",
    "    Document(page_content=\"Capital punishment refers to the death penalty in legal systems.\", metadata={\"source\": \"law\"}),\n",
    "    Document(page_content=\"东京是日本最大的城市，也是全球重要的经济中心。\", metadata={\"source\": \"news\"}),\n",
    "    Document(page_content=\"Washington D.C. is the capital of the United States.\", metadata={\"source\": \"edu\"}),\n",
    "    Document(page_content=\"首尔是韩国的首都，以现代化与传统文化的融合闻名。\", metadata={\"source\": \"travel\"}),\n",
    "    Document(page_content=\"Capitalization in finance refers to the total value of a company's shares.\", metadata={\"source\": \"finance\"})\n",
    "]\n",
    "\n",
    "query = \"各国首都城市的介绍有哪些？\"\n",
    "\n",
    "# 执行重排序\n",
    "compressed_docs = compressor.compress_documents(documents=documents, query=query)\n",
    "\n",
    "# 打印排序结果\n",
    "print(\"===== 重排序后的Top 3文档 =====\")\n",
    "for i, doc in enumerate(compressed_docs):\n",
    "    print(f\"Rank {i+1} (Score: {doc.metadata['relevance_score']:.4f}):\")\n",
    "    print(f\"内容：{doc.page_content}\")\n",
    "    print(f\"元数据：{doc.metadata}\\n{'-'*50}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 使用开源的Reranker模型"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "！modelscope download --model BAAI/bge-reranker-base --cache_dir /opt/workspace/models"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "模型路径：  /opt/workspace/models/BAAI/bge-reranker-base"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Sentence Transformers部署"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/opt/anaconda3/envs/langchain-yw/lib/python3.12/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html\n",
      "  from .autonotebook import tqdm as notebook_tqdm\n",
      "Some weights of Qwen3ForSequenceClassification were not initialized from the model checkpoint at /opt/workspace/models/Qwen/Qwen3-Reranker-0.6B and are newly initialized: ['score.weight']\n",
      "You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "=== 重排序结果 ===\n",
      "[1] Score: 0.5740 | Text: 政府宣布了新的碳减排目标，计划2030年前减少50%的碳排放...\n",
      "[2] Score: 0.5437 | Text: 最新气候法案提出建立全国性碳交易市场...\n",
      "[3] Score: 0.4945 | Text: 财政部将发行绿色债券支持环保项目...\n",
      "[4] Score: 0.3956 | Text: 交通部规划2035年全面禁售燃油车...\n",
      "[5] Score: 0.1586 | Text: 经济刺激方案包含对可再生能源企业的税收优惠...\n"
     ]
    }
   ],
   "source": [
    "from sentence_transformers import CrossEncoder\n",
    "\n",
    "# 1. 初始化Rerank模型（使用本地Qwen3-Reranker）\n",
    "model = CrossEncoder(\"/opt/workspace/models/Qwen/Qwen3-Reranker-0.6B\", device=\"cuda:3\" )\n",
    "\n",
    "# 2. 硬编码示例数据\n",
    "query = \"气候变化政策有哪些？\"\n",
    "documents = [\n",
    "    \"政府宣布了新的碳减排目标，计划2030年前减少50%的碳排放\",\n",
    "    \"经济刺激方案包含对可再生能源企业的税收优惠\",\n",
    "    \"最新气候法案提出建立全国性碳交易市场\",\n",
    "    \"财政部将发行绿色债券支持环保项目\",\n",
    "    \"交通部规划2035年全面禁售燃油车\"\n",
    "]\n",
    "\n",
    "\n",
    "# 3. 构建查询-文档对\n",
    "pairs = [(query, doc) for doc in documents]\n",
    "\n",
    "# 4. 执行重排序\n",
    "scores = [model.predict([pair])[0] for pair in pairs]  # 每次处理1个样本\n",
    "reranked_results = sorted(zip(documents, scores), key=lambda x: x[1], reverse=True)\n",
    "\n",
    "# 5. 输出结果\n",
    "print(\"=== 重排序结果 ===\")\n",
    "for rank, (doc, score) in enumerate(reranked_results, 1):\n",
    "    print(f\"[{rank}] Score: {score:.4f} | Text: {doc[:60]}...\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### HuggingFaceCrossEncoder部署"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_community.cross_encoders import HuggingFaceCrossEncoder\n",
    "\n",
    "model = HuggingFaceCrossEncoder(model_name=\"/opt/workspace/models/Qwen/Qwen3-Reranker-0.6B\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "样例"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Some weights of Qwen3ForSequenceClassification were not initialized from the model checkpoint at /opt/workspace/models/Qwen/Qwen3-Reranker-0.6B and are newly initialized: ['score.weight']\n",
      "You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "=== 重排序结果 ===\n",
      "[1] Score: 0.6017 | Text: 交通部规划2035年全面禁售燃油车...\n",
      "[2] Score: 0.5710 | Text: 最新气候法案提出建立全国性碳交易市场...\n",
      "[3] Score: 0.4596 | Text: 财政部将发行绿色债券支持环保项目...\n",
      "[4] Score: 0.3474 | Text: 经济刺激方案包含对可再生能源企业的税收优惠...\n",
      "[5] Score: 0.1195 | Text: 政府宣布了新的碳减排目标，计划2030年前减少50%的碳排放...\n"
     ]
    }
   ],
   "source": [
    "from langchain_community.cross_encoders import HuggingFaceCrossEncoder\n",
    "\n",
    "# 1. 初始化Rerank模型（使用本地Qwen3-Reranker）\n",
    "model = HuggingFaceCrossEncoder(model_name=\"/opt/workspace/models/Qwen/Qwen3-Reranker-0.6B\")\n",
    "\n",
    "# 2. 硬编码示例数据\n",
    "query = \"气候变化政策有哪些？\"\n",
    "documents = [\n",
    "    \"政府宣布了新的碳减排目标，计划2030年前减少50%的碳排放\",\n",
    "    \"经济刺激方案包含对可再生能源企业的税收优惠\",\n",
    "    \"最新气候法案提出建立全国性碳交易市场\",\n",
    "    \"财政部将发行绿色债券支持环保项目\",\n",
    "    \"交通部规划2035年全面禁售燃油车\"\n",
    "]\n",
    "\n",
    "# 3. 构建查询-文档对\n",
    "pairs = [(query, doc) for doc in documents]\n",
    "\n",
    "# 4. 执行重排序\n",
    "scores = [model.score([pair])[0] for pair in pairs]  # 每次处理1个样本\n",
    "reranked_results = sorted(zip(documents, scores), key=lambda x: x[1], reverse=True)\n",
    "\n",
    "# 5. 输出结果\n",
    "print(\"=== 重排序结果 ===\")\n",
    "for rank, (doc, score) in enumerate(reranked_results, 1):\n",
    "    print(f\"[{rank}] Score: {score:.4f} | Text: {doc[:60]}...\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "ContextualCompressionRetriever方法，结合混合检索使用"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.retrievers import ContextualCompressionRetriever\n",
    "from langchain.retrievers.document_compressors import CrossEncoderReranker\n",
    "from langchain_community.cross_encoders import HuggingFaceCrossEncoder\n",
    "\n",
    "model = HuggingFaceCrossEncoder(model_name=\"/opt/workspace/models/Qwen/Qwen3-Reranker-0.6B\")\n",
    "compressor = CrossEncoderReranker(model=model, top_n=3)\n",
    "compression_retriever = ContextualCompressionRetriever(\n",
    "    base_compressor=compressor, base_retriever=retriever\n",
    ")\n",
    "\n",
    "compressed_docs = compression_retriever.invoke(\"What is the plan for the economy?\")\n",
    "pretty_print_docs(compressed_docs)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### ollama部署"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### vllm部署 "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### xinference部署"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.13.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
